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Deflation of Eigenvalues for Iterative Methods in Lattice QCD 
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Work on generalizing the deflated, restarted GMRES algorithm, useful in lattice studies using stochastic noise 
methods, is reported. We first show how the multi-mass extension of deflated GMRES can be implemented. We 
then give a deflated GMRES method that can be used on multiple right-hand sides of Ax = b in an efficient 
manner. We also discuss and give numerical results on the possibilty of combining deflated GMRES for the first 
right hand side with a deflated BiCGStab algorithm for the subsequent right hand sides. 



1. INTRODUCTION 

Removal of the smallest eigenvalues of a matrix 
can significantly improve the performance of iter- 
ative methods. This is important for lattice QCD 
where stochastic methods for disconnected dia- 
grams and new matter formulations such as over- 
lap fermions are computationally intensive. In 
addition, there is often a need for these systems 
to be shifted to take account of multiple quark 
masses or other parameters. We think that it is 
important for any iterative method we develop to 
be applicable to general non-Hermitian systems, 
for this is often the case for lattice systems. 

GMRES is an iterative method that can 
be used in lattice studies with stochastic noise 
methods. Convergence can be improved by de- 
flating eigenvalues. Two years ago it was shown 
that deflating eigenvalues |2] can be especially 
helpful for lattice problems with multiple right- 
hand sides [H]. Here we report on generaliza- 
tions of these techniques. First we show how de- 
flated restarted GMRES can accomodate multi- 
ple shifts. We give numerical results for a Wilson 
matrix with very small eigenvalues. We then con- 
sider the case of combining multiple shifts with 
multiple right-hand sides. Finally we also discuss 
and give numerical results on the possibility of 
combining deflated GMRES for the first system 
with a BiCGStab algorithm which uses those de- 
flated eigenvalues for the subsequent solutions. 



DEFLATION FOR 
SHIFTED SYSTEMS 
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The shifted systems are expressed by 
(A - s(i)I)x l = b, 



(1) 



where the s(i) are the numerical shifts. The 
method proposed in is based upon keeping the 
residual vectors for the shifted systems parallel 
to one another. That is, one uses only a single 
Krylov subspace to solve all the systems. This is 
possible for GMRES only by forcing the residuals 
to be parallel to one another after a restart and 
thus the minimum residual condition is applied 
to only the unshifted (most difficult) system. We 
find that this also results in an almost optimal 
solution for the shifted systems. 

Our method for deflating GMRES (called 
GMRES-DR [2]) simulataneously solves linear 
equations and computes eigenvalues. This is pos- 
sible because it has a Krylov subspace for solv- 
ing the linear equations as well as Krylov sub- 
spaces designed for solving the eigenvalue equa- 
tion. These approximate eigenvectors are refined 
and improved as the GMRES-DR cycles proceed. 
We have determined that GMRES-DR can be ef- 
ficiently applied to multiply shifted systems be- 
cause of its Krylov properties. 

Fig.l shows the result of a shifted run of 
GMRES-DR(50,30), i.e., the subspace is 50- 
dimensional and includes 30 approximate eigen- 
vectors. It is compared to the residual norm 
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Figure 1. Comparison of the residual norms 
for standard (unshifted) GMRES(50) (solid mag- 
nenta) with shifted GMRES-DR(50,30). This is 
for a 12 3 x 24 Wilson-Dirac matrix at k = 0.1575 
(unshifted), 0.1552, and 0.1469. 



for regular GMRES(50). One can see the defla- 
tion start to take place around 200 matrix-vector 
products (the approximate eigenvectors are accu- 
rate enough by then). 

3. MULTIPLE SHIFTS AND ADDI- 
TIONAL RIGHT-HAND SIDES 

Typically, many right-hand sides are used to 
do stochastic estimates of matrix elements. It 
would be beneficial if the deflated eigenvalues and 
eigenvectors calculated for the first right-hand 
side could be reused. Then deflation could be- 
gin from the start instead of having to wait for 
the required eigenvectors to be recalculated, with 
more dramatic results than in Fig.l. A method 
to accomplish this called GMRES-Proj was given 
in 0. GMRES-Proj alternates cycles of GMRES 
with projections over the approximate eigenvec- 
tors. Next we consider using this for the case of 
multiple shifts for each of the multiple right-hand 
sides. 

GMRES-DR can deflate eigenvalues and han- 
dle multiple shifts at the same time for the first 
right-hand side (the right-hand sides for each 



shifted system can be kept parallel). However, for 
the second and subsequent right-hand sides there 
is no way to deflate eigenvalues by projecting over 
the previously computed approximate eigenvec- 
tors and still keep the right-hand sides for the 
different shifted systems parallel. If the eigenvec- 
tors were exact, this would be possible. So one 
approach is to use only fairly accurate approxi- 
mate eigenvectors and do a correcting iteration 
at the end for each shifted system separately. 

We now propose a better fix for this prob- 
lem which does not require accurate eigenvectors. 
GMRES-DR produces approximate eigenvectors 
that are represented in the form of an Arnoldi- 
like recurrence relation: 

AV k = V k+1 H k , (2) 

where the columns of Vk span the space of approx- 
imate eigenvectors, Vk+\ has one extra column 
added to Vk, and Hk is a (k + 1) by k matrix. 
Using this, we can perform the projection over 
the approximate eigenvectors so that the right- 
hand sides for the different shifted systems are 
all multiples of each other, except for error in 
the direction of Vk+i, the last column of Vk+i- 
This error can be easily corrected at the end, if 
we are willing to solve one additional right-hand 
side, namely Vk+i- 

There is also potential to develop a deflated 
version of BiCGStab for multiply shifted systems 
using this idea. This will be reported later, but 
for now we will look at deflating BiCGStab for 
multiple right-hand sides in the non-shifted case. 

4. DEFLATED BICGSTAB 

The deflation on the additional right-hand sides 
can employ other methods after the initial eigen- 
value generation by GMRES-DR. We have looked 
at BiCGStab, a non restarted method popular in 
the lattice community, in this context. We again 
combine a projection over approximate eigenvec- 
tors with the iterative method. However, the pro- 
jection can only be performed once before the 
BiCGStab iteration begins. Therefore this pro- 
jection must reduce the critical eigencomponents 
to a low enough level that no further reduction is 
needed during the BiCGStab iteration. The min- 
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imal residual projection is the best possible one 
in the sense that it minimizes the norm of the 
residual vector. However, it can be shown that 
even if one knows the exact eigenvectors, the cor- 
responding component in the residual vector will 
not generally be zeroed out (unless the matrix is 
Hcrmitian). To deal with this problem, we will 
give a projection that is better for this task. It 
uses both left and right approximate eigenvectors. 

Left-right Projection for deflating eigenvectors: 

1. Let the current approximate solution be 
x and the current system of equations be 
A(x — x ) = r . Let V be orthonormal with 
columns spanning the subspace of approximate 
right eigenvectors. Let U similarly be orthonor- 
mal with columns from approximate left eigen- 
vectors. 

2. Solve WAVd = c, where c = Wr . 

3. The new approximate solution is Xk = x a + 
Vd. 

This projection can reduce the small eigenvec- 
tor components better than minimum residual 
and in the case of exact left and right eigen- 
vectors, the components can be zeroed out. Of 
course, it is necessary to compute left eigenvec- 
tors. However, for many lattice problems, includ- 
ing the Wilson-Dirac matrix, there is a simple 
relationship between the left and right eigenvec- 
tors. 

Fig. 2 above shows a run on a smaller 4 4 lattice 
(using MATLAB) of the deflated BiCGStab algo- 
rithm using eigenvectors calculated from an ini- 
tial GMRES-DR solution of the first right-hand 
side. The deflated solutions take a smaller num- 
ber of iterations than the nondeflated implemen- 
tation. Notice that deflation does not care about 
the nature of the right-hand sides used, so these 
may be noise vectors or units vectors, as one de- 
sires. This applies to all techniques used. 
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Figure 2. Comparison of standard nondeflated 
BiCGStab with deflated versions using either 8 
or 16 eigenvectors from the initial GMRES-DR 
run. 
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